Semiparametric Estimation Research Articles

Policy Targeting under Network Interference

Abstract This paper studies the problem of optimally allocating treatments in the presence of spillover effects, using information from a (quasi-)experiment. I introduce a method that maximizes the sample analog of average social welfare when spillovers occur. I construct semi-parametric welfare estimators with known and unknown propensity scores and cast the optimization problem into a mixed-integer linear program, which can be solved using off-the-shelf algorithms. I derive a strong set of guarantees on regret, i.e., the difference between the maximum attainable welfare and the welfare evaluated at the estimated policy. The proposed method presents attractive features for applications: (i) it does not require network information of the target population; (ii) it exploits heterogeneity in treatment effects for targeting individuals; (iii) it does not rely on the correct specification of a particular structural model; and (iv) it accommodates constraints on the policy function. An application for targeting information on social networks illustrates the advantages of the method.

Nonparametric Estimation of Conditional Copula Using Smoothed Checkerboard Bernstein Sieves

Conditional copulas are useful tools for modeling the dependence between multiple response variables that may vary with a given set of predictor variables. Conditional dependence measures such as conditional Kendall’s tau and Spearman’s rho that can be expressed as functionals of the conditional copula are often used to evaluate the strength of dependence conditioning on the covariates. In general, semiparametric estimation methods of conditional copulas rely on an assumed parametric copula family where the copula parameter is assumed to be a function of the covariates. The functional relationship can be estimated nonparametrically using different techniques, but it is required to choose an appropriate copula model from various candidate families. In this paper, by employing the empirical checkerboard Bernstein copula (ECBC) estimator, we propose a fully nonparametric approach for estimating conditional copulas, which does not require any selection of parametric copula models. Closed-form estimates of the conditional dependence measures are derived directly from the proposed ECBC-based conditional copula estimator. We provide the large-sample consistency of the proposed estimator as well as the estimates of conditional dependence measures. The finite-sample performance of the proposed estimator and comparison with semiparametric methods are investigated through simulation studies. An application to real case studies is also provided.

Statistical inference for multivariate extremes via a geometric approach

Abstract A geometric representation for multivariate extremes, based on the shapes of scaled sample clouds in light-tailed margins and their so-called limit sets, has recently been shown to connect several existing extremal dependence concepts. However, these results are purely probabilistic, and the geometric approach itself has not been fully exploited for statistical inference. We outline a method for parametric estimation of the limit set shape, which includes a useful non-/semi-parametric estimate as a pre-processing step. More fundamentally, our approach provides a new class of asymptotically motivated statistical models for the tails of multivariate distributions, and such models can accommodate any combination of simultaneous or non-simultaneous extremes through appropriate parametric forms for the limit set shape. Extrapolation further into the tail of the distribution is possible via simulation from the fitted model. A simulation study confirms that our methodology is very competitive with existing approaches and can successfully allow estimation of small probabilities in regions where other methods struggle. We apply the methodology to two environmental datasets, with diagnostics demonstrating a good fit.

Bayesian semi-parametric estimation of compound inhomogeneous Poisson processes for ultra-high frequency financial transaction data

Marked point processes provide a flexible framework for studying ultra-high frequency financial data that records the time and price for each transaction. This paper estimates compound, inhomogeneous Poisson processes (CIPP) where trades arrive according to an inhomogeneous Poisson process, and returns are drawn from a distribution after arrival. The intensity functions have Gaussian process priors to model time-of-day (diurnal) effects, which have bursts of activity at the open and close and a midday lull, and lagged returns from the previous trade, which are idiosyncratic and depend on the stock. The nonparametric density estimator for returns imposes a shape constraint to obtain a unimodal distribution that is asymmetric and leptokurtic. A mixture model separates trades into time sensitive and time insensitive trades with different intensity functions. The latter group tends to have larger orders and higher returns. The proposed model is compared to B-splines and geometric Brownian motion.

Multistate models with nested frailty for lifetime analysis: Application to bone marrow transplantation recovery patients

Multistate models allow individuals to move through a series of states over time, making it possible to estimate the transition probabilities and intensities between states and the effect of covariates associated with each transition. Multistate models are useful for studying the evolution of individuals over time, particularly in clustered data, where individuals are grouped together and may share common risk factors. Frailties can be used in multistate models to capture the heterogeneity between groups at risk for different types of transitions, as well as the dependency structure between transitions of individuals in the same group. Frailties are unobserved random variables that are assumed to be independent between different types of transitions but shared by groups of individuals. In this paper, we propose a multistate model with frailties to capture the heterogeneity between groups at risk for different types of transitions, as well as the dependency structure between transitions of individuals in the same group. We assume that the frailties are independent between different types of transitions but shared by groups of individuals. We discuss both parametric and semiparametric estimation of the model parameters, and we evaluate the performance of the model in simulation studies. Finally, we apply the model to a dataset of bone marrow transplantation recovery patients.

SEMIPARAMETRIC ESTIMATION AND VARIABLE SELECTION FOR SPARSE SINGLE INDEX MODELS IN INCREASING DIMENSION

This paper considers semiparametric sieve estimation in high-dimensional single index models. The use of Hermite polynomials in approximating the unknown link function provides a convenient framework to conduct both estimation and variable selection. The estimation of the index parameter is formulated from solutions obtained by the routine penalized weighted linear regression procedure, where the weights are used in order to tackle the unbounded support of the regressors. The resulting index parameter estimator is shown to be consistent and sparse, and the asymptotic normality for the estimators of both the index parameter and the link function is established. To perform variable selection in the ultra-high dimension case, we further suggest a forward regression screening method, which is shown to enjoy the sure independence screening property. This screening procedure can be used before the penalized variable selection to reduce the burden of dimensionality. Numerical results show that both the variable selection procedures and the associated estimators perform well in finite samples.

Semi-parametric tensor factor analysis by iteratively projected singular value decomposition

Abstract This paper introduces a general framework of Semi-parametric TEnsor Factor Analysis (STEFA) that focuses on the methodology and theory of low-rank tensor decomposition with auxiliary covariates. Semi-parametric TEnsor Factor Analysis models extend tensor factor models by incorporating auxiliary covariates in the loading matrices. We propose an algorithm of iteratively projected singular value decomposition (IP-SVD) for the semi-parametric estimation. It iteratively projects tensor data onto the linear space spanned by the basis functions of covariates and applies singular value decomposition on matricized tensors over each mode. We establish the convergence rates of the loading matrices and the core tensor factor. The theoretical results only require a sub-exponential noise distribution, which is weaker than the assumption of sub-Gaussian tail of noise in the literature. Compared with the Tucker decomposition, IP-SVD yields more accurate estimators with a faster convergence rate. Besides estimation, we propose several prediction methods with new covariates based on the STEFA model. On both synthetic and real tensor data, we demonstrate the efficacy of the STEFA model and the IP-SVD algorithm on both the estimation and prediction tasks.

Estimating correlations among elliptically distributed random variables under any form of heteroskedasticity

The paper introduces a semiparametric estimator of the correlations among elliptically distributed random variables invariant to any form of heteroscedasticity, robust to outliers, and asymptotically normal. Our estimator is particularly fit for financial applications as vectors of stock returns are generally well approximated by heteroskedastic processes with elliptical (conditional) distributions and heavy tails. The superiority of our estimator with respect to Pearson's sample correlation in financial applications is illustrated using simulated data and real high-frequency stock returns. Using simple exponentially weighted moving averages, we extend our estimator to the case of time-varying correlations and compare it to the popular GARCH-DCC model. We show that the two approaches have comparable performances through simulations and a simple application. However, our estimator is extremely fast to compute, computationally robust, and straightforward to implement.

Semiparametric Estimation of Distribution Algorithms for Continuous Optimization

Semiparametric Estimation of Distribution Algorithms for Continuous Optimization

Semiparametric Causal Mediation Analysis with Unmeasured Mediator-Outcome Confounding

Although the exposure can be randomly assigned in studies of mediation effects, any form of direct intervention on the mediator is often infeasible. As a result, unmeasured mediator-outcome confounding can seldom be ruled out. We propose semiparametric identification of natural direct and indirect effects in the presence of unmeasured mediator-outcome confounding by leveraging heteroskedasticity restrictions on the observed data law. For inference, we develop semiparametric estimators that remain consistent under partial misspecifications of the observed data model. We illustrate the proposed estimators through both simulations and an application to evaluate the effect of self-efficacy on fatigue among health care workers during the COVID-19 outbreak.

Semiparametric estimation in generalized additive partial linear models with nonignorable nonresponse data

Semiparametric estimation in generalized additive partial linear models with nonignorable nonresponse data

Semiparametric Bayesian estimation of dynamic discrete choice models

We propose a tractable semiparametric estimation method for structural dynamic discrete choice models. The distribution of additive utility shocks in the proposed framework is modeled by location-scale mixtures of extreme value distributions with varying numbers of mixture components. Our approach exploits the analytical tractability of extreme value distributions in the multinomial choice settings and the flexibility of the location-scale mixtures. We implement the Bayesian approach to inference using Hamiltonian Monte Carlo and an approximately optimal reversible jump algorithm. In our simulation experiments, we show that the standard dynamic logit model can deliver misleading results, especially about counterfactuals, when the shocks are not extreme value distributed. Our semiparametric approach delivers reliable inference in these settings. We develop theoretical results on approximations by location-scale mixtures in an appropriate distance and posterior concentration of the set identified utility parameters and the distribution of shocks in the model.

Robust estimation of mean-variance relation.

Accurate assessment of the mean-variance relation can benefit subsequent analysis in biomedical research. However, in most biomedical data, both the true mean and the true variance are unavailable. Instead, raw data are typically used to allow forming sample mean and sample variance in practice. In addition, different experimental conditions sometimes cause a slightly different mean-variance relation from the majority of the data in the same data set. To address these issues, we propose a semiparametric estimator, where we treat the uncertainty in the sample mean as a measurement error problem, the uncertainty in the sample variance as model error, and use a mixture model to account for different mean-variance relations. Asymptotic normality of the proposed method is established and its finite sample properties are demonstrated by simulation studies. The data application shows that the proposed method produces sensible results compared with methods either ignoring the uncertainty in the sample means or ignoring the potential different mean-variance relations.

The Finite Sample Performance of Instrumental Variable-Based Estimators of the Local Average Treatment Effect When Controlling for Covariates

AbstractThis paper investigates the finite sample performance of a range of parametric, semi-parametric, and non-parametric instrumental variable estimators when controlling for a fixed set of covariates to evaluate the local average treatment effect. Our simulation designs are based on empirical labor market data from the US and vary in several dimensions, including effect heterogeneity, instrument selectivity, instrument strength, outcome distribution, and sample size. Among the estimators and simulations considered, non-parametric estimation based on the random forest (a machine learner controlling for covariates in a data-driven way) performs competitive in terms of the average coverage rates of the (bootstrap-based) 95% confidence intervals, while also being relatively precise. Non-parametric kernel regression as well as certain versions of semi-parametric radius matching on the propensity score, pair matching on the covariates, and inverse probability weighting also have a decent coverage, but are less precise than the random forest-based method. In terms of the average root mean squared error of LATE estimation, kernel regression performs best, closely followed by the random forest method, which has the lowest average absolute bias.

Causal inference of general treatment effects using neural networks with a diverging number of confounders

Semiparametric efficient estimation of various multi-valued causal effects, including quantile treatment effects, is important in economic, biomedical, and other social sciences. Under the unconfoundedness condition, adjustment for confounders requires estimating the nuisance functions relating outcome or treatment to confounders nonparametrically. This paper considers a generalized optimization framework for efficient estimation of general treatment effects using artificial neural networks (ANNs) to approximate the unknown nuisance function of growing-dimensional confounders. We establish a new approximation error bound for the ANNs to the nuisance function belonging to a mixed smoothness class without a known sparsity structure. We show that the ANNs can alleviate the “curse of dimensionality” under this circumstance. We establish the root-n consistency and asymptotic normality of the proposed general treatment effects estimators, and apply a weighted bootstrap procedure for conducting inference. The proposed methods are illustrated via simulation studies and a real data application.

A New Paradigm for High‐dimensional Data: Distance‐Based Semiparametric Feature Aggregation Framework via Between‐Subject Attributes

AbstractThis article proposes a distance‐based framework incentivized by the paradigm shift towards feature aggregation for high‐dimensional data, which does not rely on the sparse‐feature assumption or the permutation‐based inference. Focusing on distance‐based outcomes that preserve information without truncating any features, a class of semiparametric regression has been developed, which encapsulates multiple sources of high‐dimensional variables using pairwise outcomes of between‐subject attributes. Further, we propose a strategy to address the interlocking correlations among pairs via the U‐statistics‐based estimating equations (UGEE), which correspond to their unique efficient influence function (EIF). Hence, the resulting semiparametric estimators are robust to distributional misspecification while enjoying root‐n consistency and asymptotic optimality to facilitate inference. In essence, the proposed approach not only circumvents information loss due to feature selection but also improves the model's interpretability and computational feasibility. Simulation studies and applications to the human microbiome and wearables data are provided, where the feature dimensions are tens of thousands.This article is protected by copyright. All rights reserved.

A groupwise approach for inferring heterogeneous treatment effects in causal inference

Abstract Recently, there has been great interest in estimating the conditional average treatment effect using flexible machine learning methods. However, in practice, investigators often have working hypotheses about effect heterogeneity across pre-defined subgroups of study units, which we call the groupwise approach. The paper compares two modern ways to estimate groupwise treatment effects, a non-parametric approach and a semi-parametric approach, with the goal of better informing practice. Specifically, we compare (a) the underlying assumptions, (b) efficiency and adaption to the underlying data generating models, and (c) a way to combine the two approaches. We also discuss how to test a key assumption concerning the semi-parametric estimator and to obtain cluster-robust standard errors if study units in the same subgroups are correlated. We demonstrate our findings by conducting simulation studies and reanalysing the Early Childhood Longitudinal Study.

Kernel Estimation of the Quintile Share Ratio index of Inequality for Heavy-tailed Income distributions

Evidence from micro-data shows that capital incomes are exceedingly volatile, which makes up a disproportionately high contribution to the overall inequality in populations with the heavy-tailed nature on the income distributions for many countries. The quintile share ratio (QSR) is a recently introduced measure of income inequality, also forming part of the European Laeken indicators and which cover four important dimensions of social inclusion (health, education, employment and financial poverty). In 2001, the European Council decided that income inequality in the European Union member states should be described using a number of indicators including the QSR. Non-parametric estimation has been developed on the QSR index for heavy-tailed capital incomes distributions. However, this method of estimation does not give satisfactory statistical performances, since it suffers badly from under coverage, and so we cannot rely on the non-parametric estimator. Hence, we need another estimator in the case of heavy tailed populations. This is the reason why we introduce, in this paper, a class of semi-parametric estimators of theQSR index of economic inequality for heavy-tailed income distributions. Our methodology is basedon the extreme value theory, which offers adequate statistical results for such distributions. Weestablish their asymptotic distribution, and through a simulation study, we illustrate their behaviorin terms of the absolute bias and the median squared error. The simulation results clearly showthat our estimators work well.

Convolutional Neural Networks for Valid and Efficient Causal Inference

Convolutional neural networks (CNN) have been successful in machine learning applications. Their success relies on their ability to consider space invariant local features. We consider the use of CNN to fit nuisance models in semiparametric estimation of the average causal effect of a treatment. In this setting, nuisance models are functions of pretreatment covariates that need to be controlled for. In an application where we want to estimate the effect of early retirement on a health outcome, we propose to use CNN to control for time-structured covariates. Thus, CNN is used when fitting nuisance models explaining the treatment and the outcome. These fits are then combined into an augmented inverse probability weighting estimator yielding efficient and uniformly valid inference. Theoretically, we contribute by providing rates of convergence for CNN equipped with the rectified linear unit activation function and compare it to an existing result for feedforward neural networks. We also show when those rates guarantee uniformly valid inference. A Monte Carlo study is provided where the performance of the proposed estimator is evaluated and compared with other strategies. Finally, we give results on a study of the effect of early retirement on hospitalization using data covering the whole Swedish population. Supplementary materials are available online at https://github.com/stat4reg/Causal/_CNN.

Semiparametric estimation and testing for panel count data with informative interval-censored failure event.

Panel count data and interval-censored data are two types of incomplete data that often occur in event history studies. Almost all existing statistical methods are developed for their separate analysis. In this paper, we investigate a more general situation where a recurrent event process and an interval-censored failure event occur together. To intuitively and clearly explain the relationship between the recurrent current process and failure event, we propose a failure time-dependent mean model through a completely unspecified link function. To overcome the challenges arising from the blending of nonparametric components and parametric regression coefficients, we develop a two-stage conditional expected likelihood-based estimation procedure. We establish the consistency, the convergence rate and the asymptotic normality of the proposed two-stage estimator. Furthermore, we construct a class of two-sample tests for comparison of mean functions from different groups. The proposed methods are evaluated by extensive simulation studies and are illustrated with the skin cancer data that motivated this study.